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Abstract This article comments on mathematics 
textbooks for lower secondary schools. Authors do not focus 
on texts in the books but on the nonverbal elements instead. 
A possible system of categories which enables mapping and 
classifying of such elements is suggested in this article. As a 
result of that, it is possible to evaluate and compare the 
textbooks which the authors have analysed. A pilot research 
of putting this evaluation system in use is also described in 
the article. The research ascertains whether the system of 
categories is useable to obtain sets of nonverbal elements 
suitable for further analysis. 
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1. Introduction 

Recently there have been a number of publications on 
evaluation of textual content in textbooks [1-5]. There are 
also researches that focus on nonverbal elements, 
specifically in geography and civics textbooks [6-8]. Other 
researches deal with nonverbal elements from the 
multicultural and gender point of view e.g. Moraova [9] 
which we are also going to focus on in one of our following 
papers. 

As a follow-up to the researches mentioned, we present 
an article whose aim is to deal with evaluation of 
mathematics textbooks from the point of view of nonverbal 
elements. With respect to the small scope of our pilot 
research we have focused on the topic of planar and spatial 
geometry as it is defined in the framework education 
program: mathematics and its application. The specific 
chapter we have been focusing on is a chapter on 
quadrilaterals. Further in this article we will present a 
suggestion of how nonverbal elements can be categorized 
and how we can compare mathematics textbooks using this 
categorization. This would enable us to evaluate the quality 
of a textbook with respect to a criterion other than textual 
content. 


Nonverbal elements are introduced by many definitions, 
which do not vary from each other too much. A number of 
researches belonging to a research area called research on 
pictures [10-13] studied the interaction between a nonverbal 
element and the reader. We can distinguish four main 
research areas which differ in the way they look at 
nonverbal elements and the process of their interpretation. 
Janko [7] defines subareas focused on (1) the process of 
interpretation of an image; (2) the factors which effect 
remembering of images and creating mental representations 
of imaging elements; (3) research on nonverbal elements 
from the educational point of view and (4) description of 
aesthetic reactions and emotions in connection with 
nonverbal elements. 

If we focus on publications defining qualities of 
nonverbal elements from the perspective of the way they 
convey learning contents we can find a number of different 
approaches. For example Goldsmith [14], who evaluates 
nonverbal elements from the perspective of literacy, 
distinguishes three categories of nonverbal elements’ 
qualities: syntactical, semantic and pragmatic. Another 
research realized by Levin [15] focuses on the qualities of 
nonverbal elements from the perspective of cognitive 
strategies when dealing with these elements. The quality of 
a nonverbal element relates mostly to its ability to transform 
learning content into an understandable form. Levin also 
states that when it comes to cognitive processes we need to 
keep in mind that every pupil perceives a nonverbal element 
in a slightly different way based on their own cognitive 
characteristics. The nature of an educational goal, according 
to Levin, also effects the correct function of a nonverbal 
element while it’s cognitive processing. 

Ballstaedt [16], Peeck [17] or Evans [18] pay attention to 
topology of nonverbal elements with respect to specific 
qualities and determinant attributes. All these experts 
distinguish more or less similar qualities: type of nonverbal 
element, level of abstractness, relation of the element to a 
text and label of the element. Also Schallert or Levin and 
Mayer [19, 20] were concerned with a quality called 
relation of the element to a text. Apart from these, Peeck 
[17] lists other qualities of nonverbal elements, e.g. 
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aesthetic and artistic qualities, authenticity or validity. 

There are researches that deal with different aspects of 
nonverbal elements in textbooks. Approaches based on 
quantification of all types are usually used for such surveys, 
without a deeper analysis, i.e. an analysis of qualitative 
characteristics of nonverbal elements. Our long-term goal is 
to contribute to the problem of nonverbal elements as 
structural components of textbooks and to try to describe 
their function as didactic instruments. 

Pressley and Miller [21], among others, state what 
methods should be used for nonverbal elements’ evaluation. 
They claim that it is necessary to make use of various 
methods and that such research needs to be carried out 
continuously because of constant innovations in textbook 
publishing. They also mention drawbacks of some research 
methods that are often used. One of the inadequacies is a 
shallow approach resulting in shallow outcomes of the 
research. It is hence necessary to analyse nonverbal 
elements more deeply with respect to their effect on the 
learning process. Isolated evaluation of elements regardless 
their possible interconnections and relations in a given 
textbook is seen as another inadequacy. 

Particular methodological practices used for 
classification and evaluation of nonverbal elements based 
on its qualities like type of nonverbal element, level of 
abstractness, etc. (see above) are mostly of a quantitative 
character. Methodological practices used for their 
categorization are quantitative content analyses of inspected 
textbooks. Results of the researches are usually given as 
relative or absolute frequencies of the elements in given 
textbooks. It can be the quantitative character of the 
research methods that causes the shallowness of results 
which then do not cover for example an influence of 
nonverbal elements on educational situations when pupils 
or teachers work with a textbook [21]. Nevertheless, it is 
very difficult to design a research that would focus on 
educational situations. For that reason there are also 
alternative methods focusing on pupils’ and teachers’ work 
with nonverbal elements. We can mention for example 
observation, interview, verbalization methods or projective 
techniques [22-24]. 

In this article we handle nonverbal elements mostly as 
visual representations. These allow us to illustrate difficult 
mathematical phenomena in case we want to highlight only 
important features without disturbing a complicated 
structure. In the world of teaching, nonverbal elements play 
an important part in pupils’ creation of a concept of the 
given mathematical phenomenon [7]. For that reason, 
choosing a suitable set of visual representations, hence also 
textbooks is crucial. 

Nonverbal elements understood as visual representations 
can be static as well as dynamic images. Dynamic images 
cover animations or interactive geometrical shapes created 
with a computer, e.g. using the GeoGebra software. This 
article focuses only on static image which means all 
pictures in textbooks. 


Research aims 

A general aim of our research is to take part in 
clarification of nonverbal elements’ role as structural 
components of mathematics textbooks. Our findings will be 
used as a basis for an analysis of nonverbal elements as 
didactical instruments. More specifically, our research aims 
are of a methodological and contextual character. 

The methodological aim of our research is to design a 
system of categories that would allow us to classify types 
and qualities of nonverbal elements in mathematics 
textbooks and also to confirm reliability of this system. 

The contextual aim is to evaluate the potential of 
nonverbal elements in selected Czech mathematics 
textbooks to support acquiring of new learning content and 
its understanding. This aim is to be accomplished via the 
following partial aims: 

1) To carry out a comparative analysis of contemporary 
mathematics textbooks using the system of categories for 
nonverbal elements’ types and qualities evaluation (survey 
1 ). 

2) To realize interviews with pupils focusing on their 
evaluation and cognitive interaction with nonverbal 
elements in textbooks (survey 2). 

This article describes a pilot research during our survey 1, 

1. e. suggesting and altering a system of categories to 
evaluate types and selected qualities of nonverbal elements 
in mathematics textbooks and also confirming reliability of 
the system. 

With respect to our previous findings [25], which showed 
that pupils’ conceptions and mental representations of 
geometrical shapes with all necessary features are very 
insufficient, we have intentionally chosen a wide range of 
textbooks for this pilot research including a fifty-years-old 
one, whose frequency of nonverbal elements is very low. 

In our research on nonverbal elements [26] which 
focused on knowledge transfer we have discovered that 
mental representations are laden with drawbacks like 
formalism. In relation to this, another question comes to 
mind. How should we categorize nonverbal elements in 
mathematics textbooks and consequently, which categories 
are determining for forming mental representations? An 
aspect taken into account was the fact that pupils read 
without understanding and their perception of nonverbal 
elements is very superficial despite the fact that today’s 
textbooks are full of these elements. Another aim of ours 
when testing older teaching materials was to observe such 
categories which are not present in contemporary books. 

2. Materials and Methods 

The research problem and consequent research aims were 
transformed into research questions. We distinguish 
questions from the preparative phase of our research and 
questions from the phase of realization. In the preparative 
phase we set a research question: What criteria do experts 
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consider as crucial while evaluating nonverbal elements in 
mathematics textbooks? For the phase of realization we 
formulated two research questions: 

1) What types of nonverbal element do occur in 
mathematics textbooks? 

2) What qualities do these elements show? 

In the preparative phase we carried out a literature search. 
We identified which qualities of nonverbal elements effect 
pupils understanding of mathematics. We also created a list 
of characteristics of nonverbal elements in mathematics 
textbooks. As a result we obtained a first version of system 
of categories. In this phase our main interest was to find out 
how pupils perceive some of the characterized types of 
nonverbal elements. In a survey [26] we showed that pupils 
perceive some of the types incorrectly. The results of this 
survey led us to a conclusion that working with nonverbal 
elements often causes problems to children and their mental 
representations are burdened with many deficiencies 
including formalism [26]. 

2.1. System of Categories 

The system of categories we suggest in this article 
follows the system of categories designed for classification 


of nonverbal elements in geography textbooks, which was 
introduced by Janko [7]. However, we are aware of the fact 
that in mathematics we classify the nonverbal elements in a 
rather different way. In geography it is important to show 
for example photos that highlight key features or interesting 
objects that characterize a concept. On the other hand, when 
we want to highlight important features of a square, 
photography is not a very suitable nonverbal element. 
Before we start with the evaluation, we need to divide all 
nonverbal elements in mathematics textbooks into a set of 
basic types, regardless how employable the element might 
be in a lesson. 

The first two groups of nonverbal elements are dynamic 
and static nonverbal elements. As we have already 
mentioned, further on in this article we are going to focus 
on the static ones only. These we have divided into six basic 
types: geometrical shape (U), graph (G), tabular type (T), 
pictorial type (O), combined type (K) and other types (S). 
Each of these types is further subdivided into more specific 
categories. To each category there is a description to 
explain the features of corresponding nonverbal elements. 
In the legend there is an example to each category. This 
division is shown in a table, see Table 1. 


Table 1 . System of categories of nonverbal elements - first version 



category 

label 

description 

legend 

| type of nonverbal element 

dynamic 





| static 

geometrical 

shape 

precise shape 

U1 

contains important features 

right-angled triangle with highlighted 
right angle or dimensions 

drawing 

U2 

recognizable geometrical shape 

rectangular shaped bar of chocolate, 
rectangle without labels 

photography 

U3 

resembles geometrical shape 

photograph of a scaffold in a chapter 
on polygons 

graph 

graph, diagram 

G1 

curves,graphs,diagrams,sets 

bar graph of percent 

number line 

G2 

horizontal or vertical, scale 

degree scale, number line of 
integers 

tabular 

scheme 

T1 

filling triangles, squares, ... 

empty squares with results of 
multiplication 

table 

T2 

classic table 

chart of birth rate in Czech Republic 
in the period of 2000-2012 

pictorial 

drawing, picture 

01 

no mathematical phenomenon 

picture of a car in a motion word 
problem 

photography 

02 

photography of A. Einstein 

combined 

set of pictures 

K1 

set of corresponding pictures 

object zoomed in and out in a ratio, 
instructions for origami 

combination of 
pictures 

K2 

more types in one picture 

axial symmetry marked in an image 

others 

symbols, signs 

SI 

highlighted mathematical text 

graphically highlighted signs +/- 

map 

S2 

map 

map of Europe in a problem on ratio 

logo 

S3 

no geometrical meaning 

a logo on the cover of a textbook 
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2.2. Geometrical Shape (U) 


2.4. Tabular Type (T) 


This type of nonverbal elements is divided into three 
categories: precisely constructed shape, drawing and 
photography. It covers images where we can clearly 
identify geometrical shapes. 

Precise shapes with highlighted features are labelled U1. 
It can be for example a precise shape of a right angled 
triangle with marked right angle, see Figure 1(a). Shapes 
without any important features marked (right angle, sizes of 
sides, etc.) are labelled as a drawing U2. This category 
covers also shapes which are not precise or drawn by hand 
but still result in a recognizable geometrical shape, see 
Figure 1(b). The last category, photography U3, covers only 
such pictures, which contain one or more geometrical 
shapes, see Figure 1(c) [27]. 


C 




Figure 1. Examples of precise shape (a), drawing (b) and photography (c) 


2.3. Graph (G) 


This type of nonverbal elements covers all kinds of graphs 
and diagrams, category Gl, see Figure 2(a). Number lines, 
vertical as well as horizontal, and various scales are labelled 
G2. An example of a G2 category is Figure 2(b) that displays 
a scale on a thermometer in a chapter on integers. 



The tabular type of nonverbal elements includes all types 
of filling gaps of various shapes and styles, which follows a 
given rule (category Tl). A prototype example could be a 
magic square or a chart with empty boxes to fill, see Figure 
3(a). Default tables with data, which are already filled, are 
classified as T2, see Figure 3 (b). 


a 

b 

a+b 

a-b 

a-b 

a:b 

4 

0,5 






7 




0,4 

5,7 


7,6 




X 

l 

2 

3 

4 

5 

6 

7 

8 

y 

8 

4 

8 

3 

2 

8 

5 

4 

3 

8 

7 

1 


Figure 3.. Examples of filling gaps (a) and default table with data (b) 

2.5. Pictorial Type (O) 

The difference between this category of nonverbal 
elements and the category geometrical shape is that the 
pictures do not present any geometrical or mathematical 
element, which would accompany the text. An example is a 
picture of a car when students are to calculate its speed. 
Individual categories then classify only the technique used to 
create the picture (drawing or painting, photography). 

2.6. Combined Type (K) 

A combined nonverbal element is composed of more than 
one image. The K1 category covers pictures that can be 
separated but there is a connection between them. An 
example is figure with instructions for origami, see Figure 4 
(a), or a schematic overview of quadrilaterals. The K2 
category covers images that combine two or more types of 
nonverbal elements. For example photography of a butterfly 
(category 02) with marked axis of symmetry (category Ul). 
K2 also covers dimensional sketches or photography, see 
Figure 4.(b). 



Figure 2. Examples of graph (a) and vertical number line (b) 


Figure 4. Examples of separable pictures with a connection between them 
(a) and a combine of more types of nonverbal elements (b) 
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2.7. Other Types (S) 

This type of nonverbal elements is divided into three 
categories. The SI category includes highlighted 
mathematical symbols or marks, see Figure 5(a). Category 
S2 covers maps, see Figure 5(b). At last, the S3 category 
includes any other image, which does not relate to 
mathematics, e.g. a logo of a publisher, see Figure 5(c). 



Figure 5. Examples of highlighted mathematical symbols (a), map (b) and 
logo (c) 

2.8. First Pilot Research 

In our first pilot research we used these categories to 
classify nonverbal elements in four textbooks. Each time we 
focused on chapters on quadrilaterals. The aim of our 
research was only to map the nonverbal elements in these 
chapters, using our system of categories. In the research we 
have been testing whether our system of categories includes 
all types of nonverbal elements and whether all these 
categories are defined unambiguously. For that purpose we 
have asked three mathematics teachers to classify nonverbal 
elements in the four textbooks. Another aim of our pilot 
research was to compare the textbooks as far as the relative 
frequency of the nonverbal elements is concerned. Using our 
system of categories we have classified these textbooks: 

A. Molnar [28] 

B. Novotny [29] 

C. Reichel [30] 

D. Cihlaf [31 ] 

The reason for choosing these textbooks in particular was 
to discover all the possible types of nonverbal elements. 
Textbook A was published in 1999, i.e. before the 


publication of the Framework educational program. 
Textbook B was published more than 50 years ago and its 
concept is very different from contemporary textbooks. 
Book C is an Austrian textbook and textbook D was 
published in 2013 and hence follows the Framework 
educational program. There are various types present 
various frequencies present in each of these books. 

2.9. Second Pilot Research 

The results we gained in our first pilot research led us to a 
number of changes resulting in a second version of system of 
categories (see Table 2). In our second pilot research we 
evaluated a different set of textbooks and we focused on a 
different chapter called circle and disc. Book A in our second 
pilot research was published in 1997 and is intended for 
selective schools. Textbook B is from the USA and textbook 
C is intended for lower secondary schools. The reason for 
these changes was to observe whether it will affect the 
frequencies and types of nonverbal elements of our system of 
categories. The evaluators were four mathematics teachers 
who did not take part in the previous research and hence 
were not affected by it. All these changes were made to make 
it easier to test the reliability of the system of categories. The 
three following textbooks were tested in our second pilot 
research: 

A) Herman [32] 

B) Quast [33] 

C) Mullerova [34] 

There were two reasons to alternate the system of 
categories as well. The first reason was to make our system 
of categories more precise and better arranged. In our 
previous research, the “Geometrical shape” category was the 
most frequent one. For that reason we have divided it into 
three new types. On the other hand, this time we have not 
divided those categories which did not occur or occurred 
only to a small extent in the previous research. The second 
reason was a content evaluation of the potential of nonverbal 
elements in selected textbooks to support acquiring new 
educational content and help to create mental representations 
and understanding. 

The aim of our second pilot research was to confirm the 
reliability of the system of categories so that we could create 
its final version. Using this final version we can carry out a 
high-quality analysis of mathematics textbooks. 

The reliability of the system of categories was tested using 
Cohen’s kappa coefficient [35] that allows us to measure 
level of consensus between two evaluators. The value of this 
coefficient considered to be satisfying is 0.75 and more (e.g. 
Fleiss, 1981). Some authors also accept values between 0.75 
and 0.4 (e.g. Landis & Koch, 1977). The level of consensus 
for values of Kappa less than 0.4 is considered small or 
minor (e.g. Sim & Wright, 2005). 
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Table 2. System of categories of nonverbal elements - second version 



Type 

label 

description 

prototype example 

precise 

geometrical 

shape 

inquiry based 

1 

leads to autonomous work with 
the picture 

nonverbal proof of Pythagorean theorem, find 
all triangles in the picture, fill in the sizes of the 
remaining angles in the triangle 

explanation 

2 

supplies additional information 
to a text, displays prototype 
example 

image of a tangent, secant, outer line 

constructional 

3 

constructional task or its result 

solution of a problem: draw a tangent to a circle 
through given point A outside the circle 


photo/drawing 

mathematical 

4 

contains elements of the 
geometrical term which is being 
explained 

rectangular bar of chocolate, spherical ball; 
may include additional information to a 
problem 

nonmathematical 

5 

does not contain elements of the 
geometrical term which is being 
explained 

picture of a car in a motion word problem, 
picture of Pythagoras 


graph 

graph, diagram 

6 

curves, graphs, diagrams, sets 

graph of direct proportionality, pie chart, 
number sets 


tabular 

schemes 

7 

filling triangles, circles, squares 

fill the empty squares with results of 
multiplication 


others 

symbols, signs 

8 

highlighted mathematical text 

highlighted +/- 


3. Results and Discussion 

The results we gained in our first pilot research [25] can be 
seen in Figure 6, which shows the relative frequencies of 
single categories of nonverbal elements presented in 
textbooks A, B, C, D. 



A B C D 


Figure 6. Relative frequency of the first three categories of nonverbal 
elements 

They can be summed up into three conclusions: 

1) The textbooks differ from each other a lot as far as 
the “Geometrical shape” type of nonverbal elements 
is concerned. 

2) Some of the categories of our system cannot be 
easily distinguished from each other. 

3) The results handed by our evaluators were very 
diverse. 

The following study carried out in June 2015 is based on 


the results of our previous research. Our second pilot 
research was testing the new system of categories shown in 
Table 2. Our main focus was to confirm reliability of the new 
system. 

The results of our second pilot research are shown in Table 
3, Table 4 and Table 5. There is a single table for each of the 
textbooks A, B and C. The data presented there were 
collected using a content analysis method and coding. The 
results were obtained using descriptive statistics and nominal 
data analysis. Specifically, in columns called respondent 
(from 1 to 4) we display relative frequencies, which were 
calculated to determine the distribution of nonverbal 
elements in given categories. The last column displays 
Cohen’s kappa which characterises intercoder reliability of a 
given type of nonverbal elements. It is obvious from the 
results that for each type of nonverbal elements the 
evaluators reached different level of consensus when using 
the system of categories. 

Suggested system of categories designed to evaluate 
nonverbal elements and their educational potential was based 
on national and foreign researches described in the 
introduction of this article. The system was later enhanced 
with our own categories to reflect innovations in nonverbal 
components of mathematics textbooks. It turned out that 
already existing classifications cannot be simply adopted 
without any changes. Some adjustments were necessary so 
that it allowed us to evaluate nonverbal elements’ qualities. 
At the same time, it was our goal to create such system of 
categories that would be comfortable to use and that would 
not expect evaluators’ deduction. 
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Table 3. Relative frequency of single categories in textbook A 


type of 
nonverbal 
element 

textbook A 

respondent 1 

respondent 2 

respondent 3 

respondent 4 

Cohen’s 

kappa 

1 

19 

2 

27 

19 

0,48 

2 

58 

57 

40 

61 

0,65 

3 

4 

18 

14 

0 

0,41 

4 

12 

14 

9 

11 

0,57 

5 

6 

6 

7 

7 

0,73 

6 

0 

0 

0 

0 

1,00 

7 

1 

1 

1 

1 

1,00 

8 

0 

1 

0 

0 

0,75 


Table 4. Relative frequency of single categories in textbook B 


type of 
nonverbal 
element 

textbook B 

respondent 1 

respondent 2 

respondent 3 

respondent 4 

Cohen’s 

kappa 

1 

15 

0 

35 

35 

0,48 

2 

38 

24 

21 

26 

0,45 

3 

0 

29 

0 

0 

0,43 

4 

12 

9 

12 

3 

0,51 

5 

9 

24 

15 

26 

0,49 

6 

0 

0 

0 

0 

1,00 

7 

11 

11 

11 

11 

1,00 

8 

15 

0 

15 

0 

0,63 


Table 5. Relative frequency of single categories in textbook C 


type of 
nonverbal 
element 

textbook C 

respondent 1 

respondent 2 

respondent 3 

respondent 4 

Cohen’s 

kappa 

1 

0 

0 

0 

16 

0,72 

2 

42 

42 

16 

52 

0,68 

3 

35 

29 

55 

3 

0,57 

4 

10 

0 

3 

0 

0,54 

5 

10 

23 

19 

23 

0,66 

6 

0 

0 

0 

0 

1,00 

7 

3 

6 

6 

3 

0,58 

8 

0 

0 

0 

0 

1,00 


4. Conclusions 

Our aim was to show that in mathematics education it is 
important to develop pupils’ visual literacy. Visual literacy 
can be seen as a condition for working with nonverbal 
elements but also as one of the aims of education. We 
evaluate didactical elements from the point of view of their 
illustrative qualities. We see nonverbal elements as a key 
part of nontext component of textbooks. The importance of 
nonverbal elements is obvious in situations when it is 
difficult to represent the learning content verbally. 
Nonverbal elements make it possible to present learning 


contents in a way that is closer to pupils and hence also more 
understandable and acceptable. 

Given the extension of our research sample as well as the 
requirements related to reliability confirmation, we had three 
evaluators carrying out the pilot coding. These evaluators 
were instructed how to use the system of categories so that 
they were able to use it for nonverbal elements classification 
in analysed textbooks. After a successful confirmation of 
reliability the system of categories was used in the phase of 
realization. In this phase there were four evaluators using the 
system of categories for coding. It occurred that the 
evaluation of various types of nonverbal elements is a rather 
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demanding activity. We discovered that our evaluators were 
capable of focused coding for only about 60 minutes. After 
this period of time the evaluators became tired and less 
focused which might have caused lower precision of coding. 

After an analysis of empirical data obtained during 
reliability confirmation of the system of categories we came 
to a conclusion that the low level of consensus between the 
evaluators could be caused by a large number of categories 
of nonverbal elements. Based on these facts we have decided 
to design a modified system of categories. In our final 
research these changes will be done: 

The number of categories will be adjusted for the sectional 
system of categories for evaluation of the nonverbal 
elements and text connection. Together with alternations 
regarding the connection between nonverbal elements and a 
text there should also be other changes to the system to 
increase the intercoder reliability. The aim of these steps is to 
make the system more user-friendly and well-arranged. In 
general, we want our final version of system of categories to 
be a reliable research tool that can be used for mathematics 
textbooks’ evaluation and development. 

In an article Giinzel [25] we came to a conclusion that the 
system of categories needed to be specified because of some 
overlapping categories. It also needed to be adjusted to make 
it possible to confirm its reliability. Another aim resulting 
from our previous outcomes was to focus on a larger number 
of chapters and textbooks which are to be evaluated using the 
system of categories. 

The results of our second pilot research carried out with 
three different textbooks and four different mathematics 
teachers can be again summed up into three conclusions: 

1) The most frequent category of nonverbal elements in 
all textbooks is “precise geometrical shape”. The 
three types (inquiry based, explanation, 
constructional) further divide this category according 
to the function of the element in a textbook. Each of 
the types does occur in every textbook with “precise 
geometrical shape - explanation” being the most 
frequent one in all cases. 

2) If we compare the results from all four evaluators it is 
obvious that they still differ a lot in all three cases 
(textbooks A, B and C). It shows that the system of 
categories is still not reliable enough, although the 
number of evaluators as well as textbooks was too 
small to enable a deeper statistical survey. 

3) Some of the new categories are still difficult to be 
distinguished from one another which is made 
obvious in case of the three types of the first category 
“precise geometrical shape” and their frequencies in 
textbooks according to our evaluators. Another 
ambiguous set of types are those from the category 
“photography/drawing” - mathematical and 
nonmathematical type. 

The results of our research point us to yet another 
specification of our system of categories that would become 
reliable and whose types of nonverbal elements would be 


disjoint. We intend to use Cohen’s kappa [35, 36] to confirm 
the reliability of the system of categories. This test enables to 
measure the match of the evaluators using a match 
coefficient. Our research will also focus on more points of 
view of the nonverbal elements categories, e.g. a function of 
the nonverbal element in the textbook, its connection to a 
text, level of abstractness, its fidelity and the meaning of its 
label [7]. Our long-term goal is to carry out a research on 
nonverbal elements in context with understanding. A number 
of specialists points out that it is not fully relevant to draw 
conclusions about nonverbal elements’ educational potential 
based on their quantification only. Pupils apply various 
cognitive strategies when dealing with nonverbal elements 
and make use of different qualities when interpreting the 
meaning of the elements. Furthermore, we would like to find 
out how pupils evaluate the forms of nonverbal elements and 
how they understand phenomena represented by nonverbal 
elements. 
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